Tutorial 1: Automated Refinement (with BGMN)

Tutorial 1: Automated Refinement (with BGMN)#

Dara provides a Python-based wrapper to the refinement software, BGMN, which implements a robust optimization algorithm that can refine automatically in most cases. This tutorial will show you how to interact with BGMN software and how to submit, adjust, and visualize your refinements.

from pathlib import Path

from dara.refine import do_refinement_no_saving

data = Path("tutorial_data")
cif_paths = list(data.glob("*.cif"))  # include all the cif files in the data folder

pattern_fn = "CaNi(PO3)4_800_240_Ca(OH)2_(NH4)2HPO4_NiO.xy"

Basic Refinement#

A one-line refinement with the default settings.

Visualization#

You can call visualize to visualize the refinement results. The observed, calculated, and difference patterns will be plotted.

refinement.visualize()

Extracting information from the refinement#

After finishing refinement, you can read information from the RefinementResult object. The object contains the following attributes:

lst_data: information about phases, metrics of refinement (from the .lst file in BGMN)
peak_data: the simulated peaks in the calculated pattern
plot_data: x (two-theta), y_obs, y_calc, y_bkg, contribution from each phase. This is mainly used for visualization.

For example, you can get Rwp from lst_data

f"The refinement has Rwp = {refinement.lst_data.rwp} %"

'The refinement has Rwp = 5.22 %'

You can also get the information about the lattice, weight fraction for the phase.

Some values are in a tuple, with first value as the value and the second value as the error.

phase_name = "CaNi(PO3)4_15_sym"
phase_result = refinement.lst_data.phases_results[phase_name]

gewicht = phase_result.gewicht
lattice_a = phase_result.a
lattice_b = phase_result.b
lattice_c = phase_result.c
lattice_alpha = phase_result.alpha
lattice_beta = phase_result.beta
lattice_gamma = phase_result.gamma

print(f"The lattice parameters of the phase {phase_name} are:\n" \
f"    a = {lattice_a} nm, b = {lattice_b} nm, c = {lattice_c} nm,\n"  \
f"    alpha = {lattice_alpha}, beta = {lattice_beta}, gamma = {lattice_gamma}\n")
print(f"The weight fraction of the phase {phase_name} is ({gewicht[0]} ± {gewicht[1]}) %")

The lattice parameters of the phase CaNi(PO3)4_15_sym are:
    a = (1.20687, 0.0001) nm, b = (0.869142, 9.2e-05) nm, c = (0.97885, 0.00011) nm,
    alpha = None, beta = (117.9582, 0.0055), gamma = None

The weight fraction of the phase CaNi(PO3)4_15_sym is (0.2482 ± 0.0022) %

Peak data is stored in a pandas DataFrame.

refinement.peak_data

	2theta	intensity	b1	b2	h	k	l	phase	phase_idx
0	13.366635	26.976465	0.005425	1.232572e-11	1	1	0	CaNi(PO3)4_15_sym	0
1	14.293115	785.909256	0.005425	1.617005e-11	1	1	-1	CaNi(PO3)4_15_sym	0
2	16.852160	19.493126	0.005425	3.144518e-11	2	0	0	CaNi(PO3)4_15_sym	0
3	19.165318	491.529153	0.005425	5.274536e-11	1	1	1	CaNi(PO3)4_15_sym	0
4	19.667240	306.795653	0.005425	5.850873e-11	2	0	-2	CaNi(PO3)4_15_sym	0
...	...	...	...	...	...	...	...	...	...
481	43.432511	568.854649	0.004774	9.942040e-10	2	0	0	NiO_225_sym	1
482	62.967722	276.665166	0.004774	3.948112e-09	2	2	0	NiO_225_sym	1
483	75.459010	97.992894	0.004774	7.449589e-09	3	1	1	NiO_225_sym	1
484	79.437206	72.201650	0.004774	8.861775e-09	2	2	2	NiO_225_sym	1
485	95.021909	29.630559	0.004774	1.573519e-08	4	0	0	NiO_225_sym	1

486 rows × 9 columns

Refining with customized phase parameters#

The refinement with default setting looks good. But can it be better?

Dara supports the basic refinement parameters in BGMN / Profex. You can adjust the refinement parameters by passing the parameters to the do_refinement_no_saving function.

Common parameters include:

lattice_range: you can (and need to) specify the range that the lattice parameters can vary. Usually, it can be a small range, like 0.01 ~ 0.05. It is applied to all lattice parameters (a, b, c, alpha, beta, gamma).
b1: controls the width of the peak. b1 describes the average particle size in the XRD sample. The larger the b1, the broader the peak. Usually, it is constraint to a small range, like from 0 to 0.005. If the fitted b1 is too large, you will see the peaks go too broad. In this case, your simulated pattern will look like an amorphous material that can be easily fit into the background.
k1: also controls the width of the peak. k1 describes the width of the particle size distribution in the sample. The larger the k1, the smaller the distribution is. Usually, it can be constrained to 0 ~ 1.
k2: describes the microstrain in the sample. The larger the k2, the larger the microstrain. Usually, it can be a fixed value, like 0.
gewicht: means “weight” in German. it contains the information of scale factor. However, in BGMN, it can also be used to specify the preferred orientation you would like to use in the refinement. By specifying the preferred orientation, you can vary the intensity of a set of reflections in the pattern, which can help you fit your pattern better. BGMN is able to decide which reflection to adjust automatically. You only need to specify how strong the preferred orientation is. Usually, it can be SPHAR0 (none), SPHAR2 (two preferred orientation parameters), or SPHAR4 (four preferred orientation parameters), … (up to SPHAR10). The larger the gewicht, the stronger the preferred orientation is. But it can cause overfitting as well.

Input parameter format#

In Dara, all the phase parameters are passed as a dictionary arg called phase_params. The key is the parameter name, and the value is the parameter value. Dara supports three types of values:

fixed. This is a string. The parameter will be fixed to the default value (usually 0).
(initial value)_(min value)^(max value). This is a string. The parameter will be allowed to vary in the refinement between initial value to the min value to the max value. The min value begins with _, and the max value begins with ^.
Other values. It can be a string or a number. For example, setting gewicht to SPHAR2 means that preferred orientation is modeled with the SPHAR2 settings; setting lattice_range to 0.05 means that the lattice parameters can vary up to 5%.. See the BGMN / Profex manual for more information on these settings.

If you want to allow 5% variation in lattice parameters, then use the following settings. b1: (started from 0, min = 0, max = 0.005), k1: (started from 0, min = 0, max = 1), k2: fixed to 0, and gewicht: SPHAR2. This corresponds to a phase_params dict of:

phase_params = {
    "lattice_range": 0.05,
    "b1": "0_0^0.005",
    "k1": "0_0^1",
    "k2": "fixed",
    "gewicht": "SPHAR2"
}

refinement = do_refinement_no_saving(data / pattern_fn, cif_paths, phase_params=phase_params)

refinement.visualize()

Now you can see the Rwp of the refinement is slightly lower, indicating a better fit.

Specify different parameters for different phases#

In the previous example, the refinement option is applied to all phases. But you can also specify different parameters for different phases. To do so, you will need to pass a special RefinementPhase object to the phases parameter.

from dara import RefinementPhase

phases = [RefinementPhase.make(cif_path) for cif_path in cif_paths]

for phase in phases:
    # use a smaller lattice range for each phase
    phase.params["lattice_range"] = 0.01


refinement = do_refinement_no_saving(
    data / pattern_fn,
    phases=phases,
    # if one parameter is both specified in phase_params and in the RefinementPhase object, the value in RefinementPhase will be used.
    phase_params={
        "lattice_range": 0.05,  # <- this will be ignored because it has already been set in the eahc RefinementPhase object
        "b1": "0_0^0.005",
        "k1": "0_0^1",
        "k2": "fixed",
        "gewicht": "SPHAR2"
    }
)

Refining with different instrument profiles, angle range, wavelength, etc.#

If you would like to do refinement in a different instrument profiles or angle range, you can specify it in the refinement function as well.

instrument_name: the instrument profile you would like to use. The default instrument profile used is Aeris-fds-Pixcel1d-Medipix3. You can find the available instrument profiles in the dara/data/BGMN-Templates/Devices folder.
wavelength: the wavelength you would like to use in the refinement. It can be two types:
- a number: the wavelength in nm. It is useful when you analyzing the data from a synchrotron.
- a string: the element symbol. It represents the target material in the X-ray tube. BGMN can automatically find the distribution of the wavelength for the given metal. Currently, it supports sources of [“Cu”, “Co”, “Cr”, “Fe”, “Mo”]
wmin, wmax: the angle range you would like to use in the refinement. It is set in refinement_params.

refinement = do_refinement_no_saving(
    data / pattern_fn,
    cif_paths,
    instrument_name="Aeris-fds-Pixcel1d-Medipix3",
    wavelength="Cu",
    refinement_params={
        "wmin": 20,  # set the minimum two-thera in the refinement to be 20 deg.
        "wmax": 50  # set the maximum two-theta in the refinement to be 50 deg.
    }
)

refinement.visualize()

Save the project file to a folder on the disk#

Usually, you don’t have to directly interact with the BGMN input/output files.

However, if you would like to save the project directory for later use, you can use the do_refinement function instead.

The refinement files will be saved in the path specified by working_dir. Other than that, do_refinement and do_refinement_no_saving share the same parameters and output.

Feel free to modify the refinement project file yourself or by loading with the Profex software.

from dara import do_refinement

refinement = do_refinement(data / pattern_fn, cif_paths, working_dir="tutorial_refinement")

refinement_folder = Path("tutorial_refinement")

# show all the files in the folder
for file in refinement_folder.glob("*"):
    print(">", file.name)

> CaNi(PO3)4_800_240_Ca(OH)2_(NH4)2HPO4_NiO.par
> Aeris-fds-Pixcel1d-Medipix3.sav
> CaNi(PO3)4_15_sym.str
> Aeris-fds-Pixcel1d-Medipix3.geq
> Aeris-fds-Pixcel1d-Medipix3.tpl
> CaNi(PO3)4_800_240_Ca(OH)2_(NH4)2HPO4_NiO.sav
> Aeris-fds-Pixcel1d-Medipix3.ger
> CaNi(PO3)4_800_240_Ca(OH)2_(NH4)2HPO4_NiO.xy
> CaNi(PO3)4_800_240_Ca(OH)2_(NH4)2HPO4_NiO.lst
> NiO_225_sym.str
> CaNi(PO3)4_800_240_Ca(OH)2_(NH4)2HPO4_NiO.dia